cc @theacodes

Omg yes please. This will make nox much faster for common use cases and will also make tox faster for others.

We did this before and had to revert it because it broke people's setup.py files, some setup.py files make assumptions about what's on disk in certain phases of development and we cant just start YOLO excluding files.

@dstufft, what about a .pipignore solution?

ideally we should just install what's included in the source dist. I guess if someone wants the whole git repo in their source dist its their right and they should opt in to that.
(hopefully that would make them rethink their wicked ways).

That kind of patch is a bit beyond me though, I never even looked at the code for pip before.

@dstufft, also I would personally be more than happy with just .tox and .nox.
if someone relies on those they should be shot dead.

For running the Linters locally: tox -e lint-py3

https://pip.pypa.io/en/stable/development/getting-started/#running-linters

The ideal solution for this that we're looking at is to build a sdist from the current directory, and then build a wheel using that - this would avoid copying anything but the essential files needed for a build.

that would be the ideal solution.
until we get that, can we agree that excluding .tox and .nox is reasonable and is unlikely to backfire?

can we agree that excluding .tox and .nox is reasonable and is unlikely to backfire?

I'm okay with this -- but if someone comes around to asking for more, I'm gonna point them at #2195 and tell them that is how we have to solve this.

I have 0 interest in managing a blocklist that has to be kept up to date.

this may potentially break setuptool_scm worse than pip usually does

this may potentially break setuptool_scm worse than pip usually does

VCS directories would be continue to be copied.

That's not the current state of the PR but that's definitely what we're going to do, if we do this.

@RonnyPfannschmidt The original proposal not to copy VCS directories (which definitely would break setuptools_scm) or the revised one just to skip .tox and .nox?

thanks for the call out, i missed the update about the scope reduction

what about a .pipignore solution?

Well... We had #4900 and even there, I think it was clear that going dir -> sdist -> installed route is a lot better.

Well... We had #4900 and even there, I think it was clear that going dir -> sdist -> installed route is a lot better.

I agree that it makes more sense to use the sdist.
.pipignore would just be replicating logic that is needed to build the sdist anyway, seems more productive to not force users to define the sdist in two places.

Okay, with the scope reductions I think we can move forward with this?

can you point me to a test that does something similar?
install from a directory and with a place to manipulate the input directory and verify that the temp directory does not contain .tox and .nox?

it does not seem like shutil.copytree supports having an ignore glob that only effects the top level.
@chrahunt, do you have any suggestions on how to implement it without reimplementing shutil.copytree from scratch?

The ignore function receives the "current" directory being iterated over and the list of contents. If the current directory is not equal to the top-level source directory then the list of contents can be returned without any changes. E.g.

def ignore(d, names):
    if d != src:
        return names
    return [name for name in names if name not in ['.nox', '.tox']]

Could we also put a note about this new behavior somewhere in the docs? I would add it to or make a new section after pip_install/#build-system-interface.

about the doc update, I am not sure where this should go. it seems that explaining this will require explaining that pip copies the entire directory to a temp location prior to installing, which is already quiet a handful to dump on the user in the usage manual.

about the doc update, I am not sure where this should go

I'd be inclined to put it in the docs on the build system interface, as @cjerdonek suggested. It could be an additional paragraph (or subsection) explaining how a local source is prepared for building, explaining that it's copied to a temporary directory (excluding .tox and .nox subdirectories 😄) and then the build system is invoked on that temporary location.

I don't think it needs to be an in-depth discussion, just hit the points I mention above and that should do (if anyone feels like going into greater detail, they can always submit a follow-up PR).

After looking at the docs, I feel that the proper place for this is the pip_install reference page.

@pradyunsg, I feel like addressed your change request.
can you accept and land?

@pradyunsg This LGTM, but I don't want to merge with your review request outstanding. Can you confirm it's been dealt with (I think it has) and I'll merge (or you can)?

@omry Thanks for your contribution :-)

Done. Thanks again @omry!

Thanks!
Any estimate on when this will become available to end users?

In October. We have a 3 month release cadence.